Distributed application load distribution aid tool

ABSTRACT

A tool at the service of a distributed application running on machines of a distributed data processing system running in a local area network, intended for balancing the load on each of the machines of the system, includes a master daemon and a plurality of agent demons. The master and each of the agents calculate the load of the machine on which they are running. The master collects the load data of each of the agents at a first sampling interval and sends that collected load data to all of the agents. At the request of the distributed application, the local agent closest to the application indicates to the application which machine has the lightest load. The application then makes the decision to request the machine with the lightest load to execute the services the application requires. As necessary, the tool selects a master from the agents, thereby ensuring the existence and uniqueness of a master at all times, regardless of failure affecting one 1010 or more machines in the data processing system.

DISTRIBUTED APPLICATION LOAD, DISTRIBUTION AID TOOL

The present invention relates to a toolkit for balancing the load of anapplication distributed among several machines belonging to adistributed data processing system in a local area network.

FIELD OF THE INVENTION

The current trend in the development of data processing systems is toform a data processing system through the association of a plurality ofmachines connected to one another through a network, for example a localarea network. Any user can run applications of widely varied types onthis set of machines. These applications call services which supply theinformation required to handle the problem or problems they are workingon, which are offered by all or some of these machines.

When an application in the process of running requires the use of aparticular service, in current practice, it proceeds in the followingmanner:

either it chooses, in a purely random manner, the machine which willprovide it with this service and assigns the work to this machine,

or it makes a circular choice among all the machines, which means thatit assigns by turns, always in the same chronological order, the work ofproviding the services it requires in succession: thus, if the systemhas three machines, it assigns the work of providing the services itrequires first to machine No. 1, those it requires second to machine No.2, those it requires third to machine No. 3, and so on in the followingorder: No. 1, No. 2, No. 3, No. 1, etc.

No matter which of the two cases described above applies, none of thework of any of the machines is optimized in terms of time, and thecapabilities of these machines in terms of speed and performance areused only at a level far below their maximum.

There are some known solutions which make it possible to eliminate thesedrawbacks: one of these is described in French patent application No. 9408764, filed Jul. 13, 1994 by the Assignee of the subject application,under the title "Open Data Processing System with Multiple Servers". Ina system of this type, formed by the association of a central systemcalled a client with several servers, each server calculates its ownload based on the criteria specific to each application running on theclient, as well as its foreseeable development over time, and transmitsthese two factors to the client. The latter, when a particularapplication requires the services of a server, chooses the one with thelightest load during the period of time in which the services must berendered and assigns it the work of supplying the services requested.

SUMMARY OF THE INVENTION

The present invention constitutes an improvement and a generalization ofthe preceding solution.

According to the invention, the tool at the service of a distributedapplication running on the machines of a distributed data processingsystem in a local area network, which is intended for balancing the loadon each of these machines, is characterized in that it comprises aplurality of data processing modules called DAEMONs which run on thesemachines, one of which is called the master, the others being agents,

the master and the agents each having means for calculating the load ofthe machines on which they are running, at first predetermined samplinginstants, and means for storing the load data of the master and theagents,

the master containing:

means for collecting the load data of each agent, at secondpredetermined sampling instants,

means for sending the load data of each agent to all of the otheragents,

each agent containing:

means for receiving the load data of the other agents,

the local agent closest to the application indicating to the latter, atthe request of the latter, the machine with the lightest load, theapplication then making the decision to request this machine to executethe services it needs.

BRIEF DESCRIPTION OF THE DRAWING

Other characteristics and advantages of the present invention willbecome apparent from the following description given as a non-limitingexample in reference to the appended drawings. In these drawings:

FIG. 1 shows a distributed data processing system which includes theload balancing toolkit according to the invention,

FIG. 2 shows the controller for distributing the master-agent rolesamong the various elements which constitute the load balancing toolkitaccording to the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS 1) ESSENTIAL CHARACTERISTICS OFTHE TOOL ACCORDING TO THE INVENTION

A) Structure:

The various essential characteristic elements of the load balancingtoolkit ORC for (load balancing toolkit, in English) (load balancingtoolkit, in English) in a distributed data processing system accordingto the invention--for simplicity's sake, from this point forward, itwill be called the "toolkit"--appear in FIG. 1.

As shown in FIG. 1, the distributed data processing system, which can beany type whatsoever, here called SID, comprises four data processingmachines of any size and shape whatsoever, namely MCO, MCI, MC2, MC3.Each of these machines--small-, medium, or large-scalecomputers--comprise the usual elements, namely one or more centralprocessor unit, called CPU , memories associated with the latter,input/output units (I/O units,) and means for connecting to the networkRE. These means are symbolically represented by two-way arrows whichrepresent the data links between the four machines MCO through MC3 inFIG. 1.

The toolkit ORC itself comprises the master MS0 and the three agents A1,A2, A3. Any agent can also be a master, depending on conditions whichwill be explained below. Both the master and the agents are constitutedby data processing tools known to one skilled in the art as DAEMONs. ADAEMON is a data processing tool or entity running on a machine, whichis capable of responding to a question.

Inside each of the machines MCO through MC3, the DAEMONs MS0, A1, A2, A3are respectively associated with shared memories MPO, MP1, MP2, MP3.Each of them contains the load of the corresponding machine, but alsothe loads of the other machines in SID.

FIG. 1 assumes that the distributed application is running on themachine MC2 and that it requires services supplied by the other machinesMCO, MC1, MC3. This application is designated APU. The places where themaster and the agents are located are independent from the place whereAPU is running.

B) Operation:

The following are the main lines of operation of the tool ORC, it beingunderstood that during the establishment of communication between allthe machines in the system SID, it is assumed to be established that MS0is the master and that A1, A2, A3 are the agents. Refer again to FIG. 1,and specifically to the arrows and the circled numbers which accompanythem, which respectively indicate the direction of the informationflowing between the master and the agents, and the sequence ofoperations.

OPERATION 1: Each agent as well as the master collects, for the machineon which it is running, at given time intervals which constituted thefirst determined sampling instants ti, the load data of this machine,for each of the elements which constitute it (the CPU load, the loads ofthe associated memories, the loads of the I/Os, the network load, etc.).From the load of each element, expressed as a percentage of its maximumallowable load, the total load of the machine in question is calculated.This is accomplished by load calculation means, respectively MCC0 forMS0, MCC1 for A1, MCC2 for A2, MCC3 for A3. These means are simplyconstituted by calculation programs which implement the load calculationmethod described below, in paragraph 2: "Method for load calculation byeach of the agents."These means are naturally an integral part of themaster and of each agent, MS0, A1 through A3 and for this reason theyare not represented per se in FIG. 1, for simplicity's sake. Once thetotal load of the machine in question is calculated, a set ofstatistical data on the load of this machine, namely DSC, is obtained.In FIG. 1, this operation is shown for the agent A2 only, obviously forthe purpose of preserving the clarity of this figure.

OPERATION 2: At regular intervals, the agents send MS0 the statisticalload data from the corresponding machine, through the network (for A2,through the link L2 between MC2 and MC0).

OPERATION 3: The master MS0 centralizes, at practically the same regularintervals defined for Operation 2, which constitute the seconddetermined sampling instants Ti, all the statistical load data of allthe agents, as well as its own, at the level of its associated sharedmemory, in this case MP0. This centralization is in fact an operationfor collecting the load data. It is therefore executed by load datacollection means, respectively MRC0 for MS0, MRC1 for A1, MRC2 for A2,MRC3 for A3, which are in fact collection programs integrated into themaster and into each of the agents A1 through A3 and are therefore notrepresented in FIG. 1 for simplicity's sake.

OPERATION 4: The master MS0 sends, using sending means MTCO, all thisdata to each agent A1, A2, A3 through the network RE, namely through thelinks L1 between MC0 and MC1, L2 between MC0 and MC2, L3 between MC0 andMC3. MTC0 is an integral part of MS0 and is therefore not represented inFIG. 1 for simplicity's sake.

OPERATION 5: Each agent receives this load data and copies it into itsassociated shared memory, MP1 for A1, MP2 for A2, MP3 for A3. This isaccomplished by the means MRCC1 through MRCC3 for A1 through A3respectively, which means are integral parts of the latter and are notrepresented in FIG. 1 for simplicity's sake.

OPERATION 6: The application APU scans the shared memory of the machineon which it is running to search it for the load estimated for each ofthe machines, and at the moment it needs the determined services to berendered, it deduces the machine with the lightest load at this momentand requests the latter to render it these services.

2) METHOD FOR LOAD CALCULATION BY EACH OF THE AGENTS:

Examples of loads on the CPU, memory, Input/Output, and network REelements are described below.

The description of the method for load calculation by each of the meansMCCO through MCC3 is given in reference to Tables 1 through 4, whichappear below and in which the loads are given as percentages.

                  TABLE 1                                                         ______________________________________                                        Sample load data                                                              stored in any shared memory associated with an agent (expressed as a %)                   t1      t2    t3    t4  t5    t6  t7                              ______________________________________                                        CPU load (W1)                                                                             35      12    42    73  92    65  33                              Memory load (W2)                                                                          45      32    33    67  46    32  40                              Network load (W3)                                                                         12      6     33    20  12    38  5                               Input/output load (W4)                                                                    25      30    56    46  78    44  32                              ______________________________________                                    

                  TABLE 2                                                         ______________________________________                                        synthesis of series of global load data for each machine                             t1       t2    t3     t4  t5     t6  t7                                ______________________________________                                        MC0    56       32    67     63  79     82  54                                MC1    23       34    45     56  67     62  79                                MC2    32       38    34     42  35     32  36                                MC3    96       94    79     82  74     79  68                                ______________________________________                                    

                  TABLE 3                                                         ______________________________________                                        extrapolation                                                                 of the value of the global load after a time T for each machine                     t1    t2      t3  t4    t5  t6    t7  t8 = t7 + T                       ______________________________________                                        MC0   56    32      67  63    79  82    54  estimated 73                      MC1   23    34      45  56    67  62    79  estimated 82                      MC2   32    38      34  42    35  32    36  estimated 36                      MC3   96    94      79  82    74  79    68  estimated 73                      ______________________________________                                    

                  TABLE 4                                                         ______________________________________                                        application of the power coefficient, comparison and selection                                 MC0    MC1      MC2  MC3                                     ______________________________________                                        Estimated capacity                                                                             73     82       36   76                                      Power coefficient                                                                              2.5    2        0.8  1.5                                     Coefficient of the available                                                                   67.5   36       51.2 36                                      capacity                                                                      (100 - estimated load) *                                                      power coefficient                                                             ______________________________________                                    

The calculation of the load for each agent and master is identical tothat described in the above-mentioned patent. It is briefly summarizedhere.

The total load Wt of an agent (and also of the master) is obtained usingthe following formula:

    Wt=k1*W1+k2*W2+k3*W3+k4*W4, in which:

W1 is the percentage of the utilization of the central processor of theagent in terms of time.

W2 is the percentage of the utilization of the memory of the agent, thatis, the ratio between the storage capacity actually used and its totalcapacity.

W3 is the percentage of the utilization of the network by the agent,that is, the ratio between the number of pieces of information sent andreceived by the agent and the maximum allowable rate in the network.

W4 is the percentage of the utilization of the input/output units by theagent.

k1, k2, k3, k4 are specific weighting factors of the processor, thememory, the network, and the input/output units. Their sum is equalto 1. Their values depend on the nature of the application in theprocess of running, in this case APU on the machine MC2.

The loads W1, W2, W3, W4 are measured and Wt is calculated as shown ineach of the tables in Appendix 1, at determined sampling instants t1,t2, t3, t4, t5, t6, t7, etc. of the period T (which are, in fact, theinstants ti mentioned above in the description of Operation 1).

Table 1 gives an example of load data collected by any agent, forexample A1, relative to the corresponding machine MC1, for all theinstants t1through t7. This data is, of course, stored in the sharedmemory MP1 of the machine MC1 on which A1 is running, before it is sentto MS0.

This table shows, for example, that W1 is equal to 35 at the instant t1,W2 to 67 at the instant t4, W3 to 38 at t6, W4 to 32 at t7, and so on.

A calculation program API associated with APU, which runs on MC2, thenapplies--for the load data of each agent and the master which, after theexecution of Operation 3, is contained in the shared memory MP0 of MC0associated with MS0--the weighting factors k1 through k4 specific to thecorresponding machines for the application APU.

Thus Table 2 is obtained, which shows, for each of the machines MCOthrough MC4, the global load value Wt at the instants t1through t7. Thusit may be seen that, for MC0, Wt is equal to 56 at the instant t1, 32 att2, 67 at t3, etc. For MC1, Wt is equal to 23 at t1, 34 at t2, etc., andso on for the other machines.

The following step for calculating the load for all the machinesconsists of estimating, by extrapolation, using the known mathematicalmethod of least error squares, the value of the estimated load Wt at theinstant t8=(t7+T).

Thus, Table 3 is obtained. This table makes it possible to read, forexample, that the estimated values of the loads of MCO through MC3 atthis instant t8 are 73, 82, 36, and 76, respectively.

Next, a power coefficient Cp specific to each machine is applied to thetotal load of each machine in order to obtain its actual availablecapacity rate C1, using the formula:

    C1=(100-Wt(estimated))*Cp

In effect, it is important to take into account the characteristics ofeach machine, given that this is a heterogeneous data processingenvironment wherein the power, the size and the type of the machineswhich compose it are different. Thus, if a machine has a light load butat the same time does not have enough processing power to provide theservices requested of it by the application APU at a given moment, it isobvious that another machine must provide these services. Hence thenecessity for a correction factor in order to define the load, and hencethe existence of the power coefficient CP which corresponds to thispurpose.

The coefficient Cp of a given machine is calculated by generating asynthesis of the power of the central processor CPU, the capacity of thememories, the processing power of the input/output units, etc. It isrecalculated each time the hardware configuration of the machine ischanged or its operating system is modified. Likewise, each time thegeneral configuration of the distributed data processing system SID ischanged, all the coefficients CP of all the machines in the system areredefined. A Cp equal to 1 corresponds to a medium-scale machine, whichis defined by the user.

In Table 4, it is possible to read examples of actual available capacityrates C1 for each machine MCO through MC3. Thus, for MCO, with anestimated capacity rate of 73 and a power coefficient Cp of 2.5, theactual available capacity rate is 67.5. The same figures arerespectively 82, 2, 36 for MC1, and so on for MC2 and MC3.

3) METHOD FOR SELECTING THE MASTER MS0:

The basic philosophy is that any DAEMON running on any machine can be amaster. It is therefore important to develop a mechanism which makes itpossible to define which one of them will be the master and theconditions for its selection, as well as the modalities for itsreplacement should it fail.

The selection mechanism must ensure that at least 1 DAEMON is runningand that two of them cannot be masters simultaneously (especially ifthey start up at the same time).

It is composed of the following 5 main phases:

Phase 1: When a DAEMON starts up, it generates a unique identifier ID inconformity with the protocol used in the network RE, for example inconformity with the TCP-IP protocol used in the exemplary embodiment ofthe invention described herein. This identifier is composed of theEthernet address (Ethernet is the part of the TCP-IP protocol related tolocal area networks and is used in the exemplary embodiment describedherein, Ethernet being understood to be standardized and therefore knownto one skilled in the art), the moment at which the identifier istransmitted, and a random value. At the same time, it puts itself intoan intermediate state and sends these two pieces of information (itscurrent state, ID) through the network RE to all the machines in thenetwork.

Phase 2: it waits to receive the identical information from the otherDEMONS, for determined a time interval Tr (on the order of 5 to 10seconds). It is a candidate for the role of master.

Phase 3: As soon as it receives this information, it analyzes it.

If it comes from a DAEMON which is in fact a master, that is, consideredto be MS0, it considers itself to be an agent.

If it comes from a DAEMON in an intermediate state, it compares its ownidentifier with the one it receives.

If its own identifier is lower than the one it receives, it retains theright to be the master MS0.

If its own identifier is higher than or equal to the one it receives, itcedes the position. It then retransmits the two pieces of information(its own ID, its state) and again waits for responses during the timeinterval Tr, also called the timer.

Phase 4: Once this time interval has elapsed, the DAEMON in questiontries again. In order to avoid a loss of messages, which is alwayspossible in the network RE, it uses the following procedure:

The transmission and the listening for responses are repeated 5 times.

If the DAEMON in question receives a response from another DAEMON whichreveals itself to be an agent, it can be sure that a master MS0 existsand it waits for the response from the latter to reach it.

Phase 5: When 5 repetitions have taken place, and the DAEMON in questionhas not received any response from the other DAEMONs, it then decidesthat it is the master MS0.

When one of the three agents A1 through A3 realizes that the master MS0is no longer communicating with it, it starts the procedure below in allits phases, which results in the choice of a new master chosen fromamong the three.

Moreover, the master periodically notifies all the machines in thesystem SID of its existence. If the master detects the existence ofanother master, the procedure is restarted by the one whose ID is lower.

FIG. 2, which shows the controller AUT which distributes themaster-agent roles among the various DAEMONs running on the machines ofSID, will make it easier to understand the sequence of the differentphases 1 through 5 described above.

This controller AUT comprises 5 states:

State IO: The DAEMON in question sends the two pieces of information(its own ID, its state), which corresponds to Phase 1.

State I1: This DAEMON listens for responses from the other DAEMONs,which corresponds to Phases 2 and 3.

State I2: This DAEMON waits for the time interval Tr to elapse, and fora possible response from the master MS0.

State A: The DAEMON in question becomes an agent A1, A2 or A3.

State M: The DAEMON in question becomes the master MS0.

The events which correspond to this controller, which are denominated e1through e8, are the following:

e1: The DAEMON in question has disseminated its ID and its state and hasset a time interval Tr.

e2: An ID is received, and the local identifier ID (that of the DAEMONin question) is lower than the identifier it receives.

e3: An ID is received, and the local ID is greater than or equal to theID received.

e4: The time interval Tr has expired.

e5: The time interval Tr has expired and the number of attempts is lessthan 5, or an agent has just responded.

e6: The master has just responded.

e7: The time interval Tr has expired, the number of attempts is equal to5, and no agent has responded.

e8: The connection with the master is lost.

e9: Detection by a master of the existence of another master with ahigher ID.

While this invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, the preferred embodiments of the invention as set forthherein, are intended to be illustrative, not limiting. Various changesmay be made without departing from the true spirit and full scope of theinvention as set forth herein and defined in the claims.

APPENDIX

                  TABLE 1                                                         ______________________________________                                        Sample load data                                                              stored in any shared memory associated with an agent (expressed as a %)                   t1      t2    t3    t4  t5    t6  t7                              ______________________________________                                        CPU load (W1)                                                                             35      12    42    73  92    65  33                              Memory load (W2)                                                                          45      32    33    67  46    32  40                              Network load (W3)                                                                         12      6     33    20  12    38  5                               Input/output load (W4)                                                                    25      30    56    46  78    44  32                              ______________________________________                                    

                  TABLE 2                                                         ______________________________________                                        synthesis of series of global load data for each machine                      t1          t2    t3       t4  t5     t6  t7                                  ______________________________________                                        MC0    56       32    67     63  79     82  54                                MC1    23       34    45     56  67     62  79                                MC2    32       38    34     42  35     32  36                                MC3    96       94    79     82  74     79  68                                ______________________________________                                    

                  TABLE 3                                                         ______________________________________                                        extrapolation                                                                 of the value of the global load after a time T for each machine                     t1    t2      t3  t4    t5  t6    t7  t8 = t7 + T                       ______________________________________                                        MC0   56    32      67  63    79  82    54  estimated 73                      MC1   23    34      45  56    67  62    79  estimated 82                      MC2   32    38      34  42    35  32    36  estimated 36                      MC3   96    94      79  82    74  79    68  estimated 73                      ______________________________________                                    

                  TABLE 4                                                         ______________________________________                                        application of the power coefficient, comparison and selection                                 MC0    MC1      MC2  MC3                                     ______________________________________                                        Estimated capacity                                                                             73     82       36   76                                      Power coefficient                                                                              2.5    2        0.8  1.5                                     Coefficient of the available                                                                   67.5   36       51.2 36                                      capacity                                                                      (100 - estimated load) *                                                      power coefficient                                                             ______________________________________                                    

We claim:
 1. A tool (ORC) at the service of a distributed application(APU) running on machines (MC0 through MC3) of a distributed dataprocessing system (SID) in a local area network (RE), intended forbalancing a load on each of said machines, said tool comprising aplurality of data processing modules (MS0, A1 through A3) called DAEMONswhich run on said machines, one of which is a master (MS0), the othersbeing agents (A1 through A3),the master (MS0) and the agents (A1thorough A3) having respective means (MCC0 through MCC3) for calculatingat first determined sampling instants ti the load of the machines onwhich the DAEMONs are running and respective storage means (MP0 throughMP3) for storing the load data of the master and the agents, the master(MS0) containing:means (MRC0 through MRC3) for collecting load data ofthe respective agents at second determined sampling instants Ti, means(MTC0) for sending the load data of each agent to all of the agents,each agent (A1 through A3) containing:respective means (MRCC1 throughMRCC3) for receiving the load data of the other agents, whereby at arequest of the application the local agent closest to the applicationindicates to the application which machine has the lightest load, theapplication then making the decision to request said machine with thelightest load to execute services the application requires.
 2. The toolaccording to claim 1, further comprising selection means (AUT) forchoosing a master from the DAEMONs, which ensures the existence and theuniqueness of said master at the start-up of said DAEMONs and after lossof a master during the execution of the tool (ORC).
 3. The toolaccording to claim 1, further comprising means (AUT, e8) for ensuringcontinuity of the service rendered to the application by the tool incase of a failure affecting at least one machine in the data processingsystem.
 4. The tool according to claim 1, further comprising means (AUT,el, State I0) for automatically detecting from the network whichmachines will allow the network to retrieve the addresses of all themachines at the start-up of the DAEMONs.
 5. A process for implementing atool at the service of a distributed application (APU) running onmachines of a data processing system including data processing modules(MS0, A1 through A3) called DAEMONs which run on said machines, one ofwhich is a master (MS0), the others being agents (A1 through A3) theprocess comprising.1) Collecting load data for the machines at firsttime intervals ti on which the applications are running, and calculatingthe total load of each machine from said load data and from its power;2) At second time intervals Ti, sending the load data from agent dataprocessing modules (A1 thorough A3) of the corresponding machines to amaster data processing module; 3) At said second intervals Ti,centralizing the load data of the agents and the master using collectionmeans (MRC0 through MRC3); 4) Sending all of said data to each agent (A1through A3); 5) Copying all said load data from each agent into anassociated shared memory (MP1 through MP3), and 6) At the moment atwhich the application (APU) estimates it will need a particular serviceto be rendered to it, searching in shared memory (MP0 through MP3) ofthe machine on which the application (APU) is running for the estimatedload of each of the machines (MC0 through MC3), deducing the machinewith the lightest load at this moment, and then requesting said machineto render the service the application requires.
 6. A process forimplementing a tool at the service of a distributed application APU)runninig on machines of a data processing system including dataprocessing modules (MS0, A1 through A3) called DAEMONs which run on saidmachines, one of which is a master (MS0), the others being agents (A1through A3), the process comprising:1) collecting load data for machineson which the agents and the master (MS0, A1 through A3) are running, atfirst time intervals ti; 2) calculating a total load of a respectivemachine from said data and from the respective machine's power; 3)sending the master (MS0) via the respective agents (A1 through A3), atsecond time intervals Ti, the load data of a respective machine; 4)centralizing load data of all the agents as well as load data of themaster (MS0) via the master (MS0), at said second time intervals Ti; 5)sending all of said centralized load data to each agent (A1 through A3)via the master (MS0); 6) copying all said centralized load data intorespective shared memories (MP1 through MP3) by each agent (A1 throughA3); and 7) performing the following steps by the application (APU) at amoment at which the application (APU) estimates the application willneed a particular service to be rendered:a) searching a shared memory(MP0 through MP3) of a machine on which the application (APU) is runningfor an estimated load of each of the machines (MC0 through MC3), b)deducing a machine with the lightest load at said moment, and c)requesting said machine with the lightest load to render the application(APU) said service.
 7. The invention as set forth in claim 6 wherein thestep of calculating a total load is performed by a calculation program.8. The invention as set forth in claim 7 wherein the step ofcentralizing load data is performed by a collection program.
 9. Theinvention as set forth in claim 7 wherein the step of sending all ofsaid centralized load data is effected by an integral part (MTC0) ofsaid master (MS0).
 10. The invention as set forth in claim 8 wherein thestep of sending all of said centralized load data is effected by anintegral part (MTC0) of said master (MS0).
 11. The invention as setforth in claim 9 wherein the step of sending all of said centralizedload data is effected by an integral part (MTC0) of said master (MS0).12. The invention as set forth in claim 9 wherein the step of sendingall of said centralized load data is effected by an integral part (MTC0)of said master (MS0).
 13. The invention as set forth in claim 6 whereinthe step of centralizing load data is performed by a collection program.14. The invention as set forth in claim 13 wherein the step ofcentralizing load data is performed by a collection program.
 15. Theinvention as set forth in claim 13 wherein the step of sending all ofsaid centralized load data is effected by an integral part (MTC0) ofsaid master (MS0).
 16. The invention as set forth in claim 14 whereinthe step of sending all of said centralized load data is effected by anintegral part (MTC0) of said master (MS0).
 17. The invention as setforth in claim 6 wherein the step of sending all of said centralizedload data is effected by an integral part (MTC0) of said master (MS0).18. The invention as set forth in claim 17 wherein the step ofcentralizing load data is performed by a collection program.
 19. Theinvention as set forth in claim 17 wherein the step of sending all ofsaid centralized load data is effected by an integral part (MTC0) ofsaid master (MS0).
 20. The invention as set forth in claim 18 whereinthe step of sending all of said centralized load data is effected by anintegral part (MTC0) of said master (MS0).